METHOD AND SYSTEM FOR COMPARISON-BASED PRIORITIZED BIT 

RATE CONVERSION 

FIELD OF THE INVENTION 

[0001] The present invention relates to communication methods and systems 
in 'general, and to methods and systems for comparison-based bit rate 
conversion of compressed media. 

BACKGROUND OF THE INVENTION 

[0002] Methods and systems for compressing and transmitting media signals 
are known in the art. Compressed digital video is largely becoming the preferred 
medium to transmit to video viewers everywhere. Part of the Moving Pictures 
Experts Group (MPEG) specifications are standardized methods for compressing 
and transmitting video. Various audio compression techniques are also known in 
the art. In general, MPEG is used today for transmitting video over terrestrial, 
wireless, satellite and cable communication channels and also for storing digital 
video. 

[0003] An audio stream is organized as an ordered sequence of frames. A 
video stream is usually organized as an ordered sequence of pictures, each 
picture includes a plurality of pixels. A video picture includes a plurality of slices, 
each slice including a plurality of macro blocks. The audio and video streams are 
provided to an audio encoder and video encoder respectively to generate 
compressed audio and video elementary streams, also referred to as elementary 
streams. 

[0004] MPEG compression/encoding utilizes various compression schemes, 
such as adaptive quantization, intra-frame encoding, inter-frame encoding, run 
length encoding and variable length coding. Intra-frame coding takes advantage 
of spatial redundancies in a picture. Inter-frame coding takes advantage of 
temporal redundancies from picture to picture in a video sequence. Inter-frame 
coding involves motion estimation and motion compensation. There are three 
types of motion estimations - forward, backward and bi-directional. Macroblocks 
are the elementary unit for motion compensation and adaptive quantization. Each 
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macroblock is associated with a quantization factor field, representative of the 
degree of quantization. A slice, including a plurality of macroblocks includes a 
slice header that has a quantization factor field that is associated to some of the 
macro blocks of the slice. 

[0005] The compressed elementary streams usually include a sequence of 
three types of pictures. These types are known as l-picture, P-picture and B- 
picture. l-pictures use only intra-coding. P-pictures use forward prediction and 
usually also intra-coding. B-pictures use bi-directional coding (forward and/or 
backward prediction) and optionally also intra-coding. In a sequence of I, P, and 
B-pictures, each P-picture is encoded in view of a previous l-picture or P-picture. 
Each B-picture is coded using a previous l-picture of P-picture and/or a next I- 
picture or P-picture. 

[0006] A recognizable picture can be reconstructed from an l-picture alone, 
but not from a B-picture alone. Only l-pictures and P-pictures can be anchor 
pictures that are used to predict another pictures, l-pictures allow for 
reconstructing a recognizable picture but offers only relatively moderate 
compression. B-pictures are usually much smaller than l-pictures. Each picture 
includes a picture header that includes a picture type indication, indicating 
whether the picture is an I, B or P picture. 

[0007] Pictures are sometimes arranged in groups, that are referred to as 
Group Of Pictures (GOP). Usually, each GOP starts by an l-picture that is 
followed by B-pictures and P-pictures. 

[0008] Elementary streams are packetized to produce PES packets. PES 
packets made up of elementary streams that form a program share a common 
time base. The PES packets may also include additional information. PES 
packets of distinct elementary streams can be arranged as either a Program 
Stream or a Transport Stream. At least one or more stream of PES packets 
having a common base time are usually combined to a Program Stream. A 
Transport Stream combines one or more programs with one or more 
independent time bases into a single stream. Transport Streams include 
transport packets of 188 bytes. Transport Stream packets start with a transport 
packet header. The header includes a packet ID (PID). Transport Stream packets 
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of one PID value carry data of a single elementary stream. Transport Streams 
include Program Specific Information (PSI) tables. The PSI tables specify which 
PIDs and accordingly which elementary streams are associated to form each 
program. 

[0009] Transport Streams may be of either fixed or variable bit rate. Some 
programs of the Transport Stream are of a variable bit rate, if, for example, more 
bits are allocated to complex scenes, and less bits are allocated to more simple 
scenes. 

[0010] Transport Streams are provided to a channel of a limited available 
bandwidth/ storage space. The ISO/IEC 13818-1 specification defines a channel 
as a digital medium that stores or transports a Transport or a Program Stream. 
The aggregate bandwidth of all the components of the Transport Stream must 
not exceed, at any time, the available bandwidth of the channel. 
[0011] Various lossy and lossless techniques are implemented to adapt the 
aggregate bandwidth of the programs of a Transport Stream to the available 
bandwidth of a channel. U.S. patents 6,038,256 and 6,192,083 of Linzer et al, 
U.S. patents 5,862,140 and 5,956,088 of Shen et al and U.S patent 5,877,812 of 
Krause et al, describe some of these prior art methods. Lossless techniques, 
such as statistical multiplexing, do not require further compressing of media 
pictures. Lossless techniques also include delaying or advancing a transmission 
of transport packets. Lossy techniques involve additional compression, and are 
usually implemented whenever the appliance of lossless techniques is not 
feasible or does not provide sufficient results. The further compression usually 
results in visual quality degradation. 

[0012] Some prior art methods base their compression decisions upon a 
complexity of a scene. A disadvantage of these prior art methods is that they are 
at most adapted to perform a first modification (such as compression) of a media 
stream and are not suited to perform additional modifications (such as 
recompression) of a media stream. Another disadvantage of some prior art 
methods is that these methods contribute to a quality fluctuation along a stream. 
[0013] Some prior art systems, such as Rhode & Schwartz digital video 
quality analyzer DVQ and Tektronix quality of service monitor PQM300 allow for 
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measuring the quality of a video picture. Each DVQ is configured to measure the 
quality of one picture at a time. Measuring the quality of multiple programs within 
a transport stream requires a plurality of DVQ, as illustrated in the article 
"Statistical multiplex - what does it mean for DVB-T?" by Dr. Kuhn and Dr, 
Antkowiak, FKT Fachezeitschrift fur Ferensehen, Film und elektronische Medien 
April 2000, reprinted in http://www.rhodeschwarts.com. Multiple PQM300 are 
required to monitor a plurality of programs. As the DVQ and the PQM300 are 
relatively expensive, real time measurements of multiple programs within a single 
transport stream is very costly. 

[0014] Another disadvantage of the mentioned above prior art methods is that 
they cannot be tuned/controlled/ refined in view of external information such as 
video provider preferences, viewers preferences or additional information, such 
as quality or quality degradation statistics. 

[0015] There is a need to provide a system and a method for providing a 
multiplexed sequence, the multiplexed sequence including at least one sequence 
of basic media data units and/or replacing basic media data units, the system 
and method are responsive to at least one characteristic (such as quality, quality 
degradation, compression level and the like) of at least some of the basic media 
data units. 

[0016] There is a need to provide a system and a method for providing a 
multiplexed sequence whereas the basic media data units of the multiplexed 
sequence are characterized by either an optimal quality, optimal quality 
degradation, optimal compression level, or a combination of said characteristics. 
[0017] There is a need to provide a system and a method for adaptation of the 
aggregate bandwidth of the programs of a Transport Stream to the available 
bandwidth of a channel that provides programs with sufficient quality. 
[0018] There is a need to provide a system and a method for adaptation of the 
aggregate bandwidth of the programs of a Transport Stream to the available 
bandwidth of a channel that are responsive to the quality degradation of each 
program. 

[0019] There is a need to provide a system and a method for adaptation of the 
aggregate bandwidth/bit-rate of the programs of a Transport Stream to the 
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available bandwidth of a channel that reduces compression level fluctuations 
and/or quality fluctuations in encoded video programs. 

[0020] There is a need to provide a system and a method for applying lossy 
techniques for adaptation of the aggregate bandwidth of the programs of a 
Transport Stream to the available bandwidth of a channel in response to the 
compression level of basic media data units, such as macroblocks. 
[0021] There is a need to provide a system and a method for adaptation of the 
aggregate bandwidth of the programs of a Transport Stream to the available 
bandwidth of a channel that provide an optimal Transport Stream, an optimal 
Transport Stream being characterized by optimal quality, compression level, 
quality degradation or a combination of said parameters. 
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SUMMARY OF THE PRESENT INVENTION 

[0022] The invention provides a system and a method for providing a 
multiplexed sequence, that are responsive to at least one characteristic (such as 
quality, quality degradation, compression level and the like, or a combination of 
at least two of the characteristics) of at least some of the basic media data units. 
[0023] The invention provides a system and a method for providing a 
multiplexed sequence including at least one sequence of basic media data units, 
modified basic media data units and/or selected modified basic media data units, 
whereas the basic media data units of the multiplexed sequence are 
characterized by either an optimal quality, an optimal quality degradation, or an 
optimal compression level, or a combination of said characteristics. 
[0024] The invention provides a system and a method for adaptation of the 
aggregate bandwidth of the programs of a Transport Stream to the available 
bandwidth of a channel that provides programs with sufficient quality. 
Conveniently, the system and method are responsive to the quality degradation 
of each program. 

[0025] The invention provides a system and a method for adaptation of the 
aggregate bandwidth of the programs of a Transport Stream to the available 
bandwidth of a channel that reduces compression level fluctuations and/or quality 
fluctuations in encoded video programs. 

[0026] The invention provides a system and a method for applying lossy 
techniques for adaptation of the aggregate bandwidth of the programs of a 
Transport Stream to the available bandwidth of a channel in response to the 
compression level of basic media data units, such as macroblocks. 
[0027] The invention provides a system and a method for adaptation of the 
aggregate bandwidth of the programs of a Transport Stream to the available 
bandwidth of a channel that provides an optimal Transport Stream. An optimal 
Transport Stream being characterized by either optimal quality, compression 
level, quality degradation or a combination of said parameters. 
[0028] The invention provides a system and a method for providing a 
multiplexed sequence including a plurality of basic media data units and a 
plurality of modified basic media data units. The modification is performed in 
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response to a modification priority and optionally in response to a target size of 
the multiplexed sequence. The parameters can include quality, quality 
degradation, compression level, external modification information and the like. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

[0029] The present invention will be understood and appreciated more fully 
from the following detailed description taken in conjunction with the drawings in 
which: 

[0030] Figure 1 , is a schematic description of a statistical multiplexer and its 
environment, in accordance to a preferred embodiment of the invention; 
[0031] Figure 2 is a schematic description of a plurality of statistical 
multiplexers coupled to a plurality of sets of service groups, constructed and 
operative in accordance with preferred embodiments of the present invention; 
[0032] Figure 3 is a flow chart diagram of a method for generating a 
multiplexed sequence of basic media data units and modified basic media data 
units to be transmitted over a communication medium; 

[0033] Figure 4 is a flow chart diagram illustrating a method for generating 
and transmitting a multiplexed sequence over a communication channel, in 
accordance with a preferred embodiment of the invention; 
[0034] Figure 5 illustrates a plurality of received basic media data units and a 
content of a multiplexed sequence, in accordance to a preferred embodiment of 
the invention; and 

[0035] Figure 6 is a schematic description of a statistical multiplexer, in 
accordance to a preferred embodiment of the invention. 
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DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 
[0036] It should be noted that the particular terms and expressions employed 
and the particular structural and operational details disclosed in the detailed 
description and accompanying drawings are for illustrative purposes only and are 
not intended to in any way limit the scope of the invention as described in the 
appended claims. 

[0037] The invention provides a method for generating a multiplexed 
sequence, the method including the steps of: (a) Receiving at least one basic 
media data unit sequence. (P) Determining modification priorities of a plurality of 
basic media data units of the received at least one basic media data unit 
sequence. The plurality can include all the received basic media data units but 
this is not necessary, (y) Selecting basic media data units to be modified, in 
response to the modification priority of each basic media data unit. (5) Modifying 
each of the selected basic media data units to provide corresponding modified 
basic media data units, (e) Replacing selected basic media data units with the 
corresponding modified basic media data units in response to a comparison 
between the basic media data units and the corresponding modified basic media 
data units to provide replacing basic media data units. (0 Multiplexing replacing 
basic media data units and basic media data units that were not replaced to 
provide the multiplexed sequence. 

[0038] The invention provides a method for generating and transmitting a 
multiplexed sequence over a communication channel, the communication 
channel has an available bandwidth, the method including the steps of: (/) 
Receiving at least one basic media data unit sequence. (//) Determining a 
modification priority of a plurality of basic media data unit of the received at least 
one basic media data unit sequence. (Hi) Selecting basic media data units to be 
modified, in response to the modification priority and to the available bandwidth. 
(/V) Modifying each of the selected basic media data units to provide 
corresponding modified basic media data units, (v) Replacing selected basic 
media data units with the corresponding modified basic media data units in 
response to a comparison between the basic media data units and the 
corresponding modified basic media data units to provide replacing basic media 
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data units, (w) Multiplexing replacing basic media data units and basic media 
data units that were not replaced to provide the multiplexed sequence. 
[0039] According to an aspect of the invention the modification priority of a 
basic media data unit is responsive to the compression level of the basic media 
date unit. Conveniently, the modification priority is inversely proportional to the 
compression level of the basic media data unit. Preferably, a compression level 
threshold is determined and basic media data units that have a higher 
compression level are not transmitted or modified. If a modification of a basic 
media data unit results in a corresponding modified basic media data unit of 
higher compression level than the threshold, the corresponding modified basic 
media data unit is not transmitted or applied to the multiplexed sequence. 
Conveniently, the compression level is learnt from a compression indication, 
such as but not limited to a quantizing value. 

[0040] According to another preferred embodiment of the invention, a basic 
media data unit can be converted to at least two corresponding modified basic 
media data units, and the method also includes a step of determining whether to 
replace the basic media data unit and by which corresponding modified basic 
media data unit of the at least two corresponding basic media data units. 
[0041] According to an aspect of the invention the modification priority of a 
basic media data unit is responsive to the quality of the basic media data unit or 
to a combination of qualities of basic media data units either associated with the 
basic media data unit or grouped together with the basic media data unit. 
Conveniently, the modification priority is proportional to the quality of the basic 
media data unit. Preferably, a quality threshold is determined and basic media 
data units that have a lower quality are not modified. If a modification of a basic 
media data unit results in a corresponding modified basic media data unit that 
has a quality that is below the quality threshold, the corresponding modified basic 
media data unit is not transmitted or appended to the multiplexed sequence. 
Conveniently, the quality of portions of the multiplexed sequence can be 
measured by quality measurement units or from can be learnt from a 
compression indication, such as but not limited to a quantizing value. 
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[0042] According to an aspect of the invention the modification priority of a 
basic media data unit is responsive to the quality degradation of the basic media 
data unit or to a combination of quality degradation of basic media data units 
either associated with the basic media data unit or grouped together with the 
basic media data unit. The quality degradation can indicate a quality loss 
resulting from either additional or previous modification, compression or 
alteration of a basic media data unit or from the generation of the basic media 
data unit. Conveniently, the modification priority is inversely proportional to the 
quality degradation of the basic media data unit. Preferably, a quality degradation 
threshold is determined and basic media data units of higher quality degradation 
are not transmitted or modified. If a modification of a basic media data unit 
results in a corresponding modified basic media data unit that is characterized by 
quality degradation that is above the quality degradation threshold, the 
corresponding modified basic media data unit is not transmitted or appended to 
the multiplexed sequence. Conveniently, the quality degradation of portions of 
the multiplexed sequence is measured by quality measurement units. According 
to another aspect of the invention the quality degradation is learnt from signal to 
noise measurements or from a compression indication, such as but not limited to 
a quantizing value. 

[0043] According to an aspect of the invention the comparison involves 
comparing the qualities of the basic media data unit and the corresponding 
modified basic media data unit. 

[0044] According to an aspect of the invention, media data units are arranged 
in groups and the comparison involves comparing a combination of qualities of 
basic media data units belonging to the group and of corresponding basic media 
data units. 

[0045] According to an aspect of the invention, the comparison involves 
comparing the compression levels of the basic media data unit and the 
corresponding modified basic media data unit. 

[0046] According to an aspect of the invention basic media data units are 
arranged in groups and wherein the comparison involves comparing a 
combination of compression levels of basic media data units belonging to the 
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group and of corresponding basic media data units. The comparison may involve 
the temporary storage of basic media data units of a group and of the 
characteristics of the members of the group. 

[0047] According to an aspect of the invention at least some of the basic 
media data units include temporal difference information representative of 
temporal differences between basic media data units belonging to the same 
basic media data unit sequence; and wherein the comparison involves 
comparing an amount of temporal difference information within the basic media 
data unit and the corresponding modified basic media data unit. 
[0048] According to an aspect of the invention the comparison between a 
basic media data unit and a corresponding modified basic media data unit can 
include comparing between a combination of at least two of the following 
parameters: (a) quality; (b) combination of qualities of basic media data unit 
group members; (c) compression level; (d) combination of compression levels of 
basic media data unit group members; (e) temporal difference information; (f) 
combination of temporal difference information of basic media data unit group 
members; and (g) size of the basic media data unit and the corresponding basic 
media data unit. 

[0049] According to another aspect of the invention the replacement of 
selected basic media data units by corresponding basic media data units is 
responsive to a comparison between a basic media data unit, a corresponding 
modified basic media data unit; another basic media data unit; another 
corresponding basic media data unit. The comparison can include comparing 
between a combination of at least two of the following parameters: (a) quality; (b) 
combination of qualities of basic media data unit group members and 
combination of qualities of other basic media data unit group members; (c) 
compression level; (d) combination of compression levels of basic media data 
unit group members and combination of compression levels of other basic media 
data unit group; (e) temporal difference information; (f) combination of temporal 
difference information of basic media data unit group members and combination 
of temporal difference information of other basic media data unit group members; 
and (g) size. 
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[0050] According to an aspect of the invention the system and method 
provides a multiplexed sequence whereas the basic media data units of the 
multiplexed sequence are characterized by either an optimal quality, optimal 
quality degradation, optimal compression level, or a combination of said 
characteristics. 

[0d51] The at least one characteristic of a received basic media data unit can 
be changed to an optimal value, by setting the modification priority to reflect said 
parameter, by setting allowable ranges of said parameter, and by selecting either 
a basic media data unit or a corresponding modified basic media data unit in 
response to the proximity of their characteristic to the optimal characteristic. 
[0052] The invention provides a statistical multiplexer for providing a 
multiplexed sequence including at least one basic media data sequence, the 
statistical multiplexer including: (i) A control unit, (ii) At least one input, coupled to 
the control unit, for receiving at least one basic input data unit sequence, (ni) An 
output, coupled to the control unit and to a communication module, for providing 
a multiplexed sequence to a communication module; the communication channel 
has an available bandwidth, (jy) A modification unit, coupled to the control unit, to 
the at least one input and to the output, the modification unit is configured to 
modify selected basic media data units to provide corresponding basic media 
data units, in response to control signals from the control unit; wherein a modified 
selected basic media data unit is smaller than the corresponding selected basic 
media data unit. The control unit is configured to: (U) determine modification 
priorities of a plurality of basic media data units out of the received at least one 
basic media data unit sequence; (iji) select basic media data units to be 
modified, in response to the modification priority and to an available bandwidth of 
the communication module; ( i.iii ) control the provision of the selected basic media 
data units to the modification unit and the generation of corresponding modified 
basic media data units, ( i.iv ) control the replacement of selected basic media 
data units with the corresponding modified basic media data units in response to 
a comparison between the basic media data units and the corresponding 
modified basic media data units to provide replacing basic media data units. (Ly) 
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control the provision of a multiplexed sequence including replacing basic media 
data units and basic media data units that were not replaced. 
[0053] According to an aspect of the invention, the modification priority of a 
basic media data unit is further responsive to an amount or even to an existence 
of temporal difference information within the basic media data unit. 
[0054] According to an aspect of the invention, the modification priority of a 
basic media data unit is further responsive to a dependency of at least one other 
basic media data unit upon the basic media data unit. For example, in MPEG 
compliant video streams, basic media data units such as macroblocks of I- 
pictures will have a lower modification priority than macroblocks of P- pictures, 
and macroblocks of P-pictures will have a lower modification priority than 
macroblocks of B-pictures. As l-pictures and P-pictures can act as anchor 
pictures of other pictures. 

[0055] According to an aspect of the invention the modification priority of a 
basic media data unit is responsive to a combination of at least two of the 
following parameters: (a) a quality of the basic media data unit; (b) a quality 
degradation of the basic media data unit; (c) a compression level of the basic 
media data unit; (d) a dependency of at least one other basic media data unit 
upon the basic media data unit; (e) a combination of qualities of basic media data 
units that belong to a same basic media data unit group as the basic media data 
unit; (f) a combination of quality degradations of basic media data units that 
belong to a same basic media data unit group as the basic media data unit; (g) a 
combination of compression levels of basic media data units that belong to a 
same basic media data unit group as the basic media data unit; (h) size of the 
basic media data unit; (i) dependency of at least one other basic media data unit 
on the basic media data unit; (j) combination of dependencies of other basic 
media data units on basic media data units that belong to the same basic media 
data unit group as the basic media data unit. Conveniently, each one of said 
parameters is associated with a predefined parameter allowable range, such that 
basic media data units that are characterized by a parameter that is out of the 
range are not modified, transmitted or appended to the multiplex. 
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[0056] According to an aspect of the invention, each sequence of basic media 
data units is distinguishable from at least one other sequence of basic media 
data units. The modification priority of each basic media data unit belonging to a 
basic media data unit sequence reflects the identity of the basic media data unit 
sequence. The modification priority can be further responsive to at least one of 
the mentioned above parameters (a)- (j). 

[0057] According to an aspect of the invention, each sequence of basic media 
data units is to be provided to a corresponding buffer, wherein the modification 
priority of each basic media data unit of a sequence is responsive to a simulated 
simulated status of the corresponding buffer. The modification priority can be 
further responsive to at least one of the mentioned above parameters (a)- (j). 
[0058] The invention provides a method wherein the modification priority is 
responsive to external modification priority information. The external priority 
modification information, can be provided by various entities located along a 
logical path extending from a generating entity of the basic media data units, at 
least one multiplexing or even re-multiplexing entity, to end-users of the 
sequences of the basic media data units. The modification priority can be further 
responsive to at least one of the mentioned above parameters (a)- Q). The 
external modification priority information reflects at least one of the following 
parameters: (a1 ) at least one end-user's preferences; (b1 ) at least one end- 
user's profile, (c1) at least one end-user's behavior pattern, (d1) at least one 
media provider's preferences, (e1) at least one media provider profile. 
[0059] Conveniently, the step of selecting basic media data units to be 
modified, is preceded by a step of applying at least one lossless technique for 
generating the multiplexed sequence such that the amount of selected basic 
media data unit modifications is reduced. 

[0060] Conveniently, a basic media data unit includes a plurality of media 
signals, such as MPEG compliant signals; JPEG compliant signals; M-JPEG 
compliant signals; video signals; audio signals; data signals; H.261 compliant 
media signals; H.263 compliant signals; streaming media signals; high quality 
audio signals; AC-3 audio signals; and AAC audio signals. If original media 
signals, such as analog or digitized media signals are received the method 
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further requires to compress the original media signals to provide compressed 
media signals. Preferably, a basic media data unit is either a group of pictures, a 
picture, a slice, a macroblock, or a sequence of macroblocks. 
[0061] According to yet an aspect of the invention the method further includes 
a step of transmitting the multiplexed sequence over a communication channel 
having an available bandwidth and wherein the bandwidth of the multiplexed 
sequence does not exceed the available bandwidth. 
[0062] According to an aspect of the invention the system and method 
provides a multiplexed sequence whereas the basic media data units of the 
multiplexed sequence are characterized by either a optimal quality, optimal 
quality degradation, optimal compression level, or a combination of said 
characteristics. The at least one characteristic of a received basic media data 
unit can be changed to an optimal value, by setting the modification priority to 
reflect said parameter, and setting allowable ranges of said parameter. 
[0063] According to an aspect of the invention, setting a modification priority 
to be proportional to a characteristic (such as quality, quality degradation, 
compression level) of a basic media data unit results in a reduction of 
variance/range of that characteristic in the multiplexed sequence. 
[0064] According to another aspect of the invention the modification priority of 
a basic media data unit is determined in accordance with a modification priority 
function. Conveniently, the modification priority function can be changed 
dynamically. Preferably, the modification priority function itself is responsive to at 
least one characteristic of at least a portion of received basic media data units. 
The characteristics can be either a total amount of basic media data units 
received during a predefined time period, the amount of basic media data unit 
within at least one sequence of basic media sequence, which basic media data 
sequences are received, the quality of at least some of the received basic media 
data units, the quality degradation of at least some of the received basic media 
data units, the compression level of at least some of the basic media data units, 
and the like. The modification priority function can also be responsive to at least 
one of the following parameters: amount of received basic media data units; 
amount of basic media data units belonging to predefined basic media data unit 
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sequences; reception rate of received basic media data units; reception rate of 
basic media data units belonging to predefined basic media data sequences; and 
the identity of received basic media data unit sequences. 
[0065] According to another aspect of the invention the comparison is 
executed by applying a comparison function. Conveniently, the comparison 
function can be changed dynamically. Preferably, the comparison function itself 
is responsive to at least one characteristic of at least a portion of received basic 
media data units. The characteristics can be either a total amount of basic media 
data units received during a predefined time period, the amount of basic media 
data units within at least one sequence of basic media data unit sequence, the 
identity of the received basic media data sequences, the quality of at least some 
of the received basic media data units, the quality degradation of at least some of 
the received basic media data units, the compression level of at least some of 
the basic media data units, and the like. The comparison function can also be 
responsive to at least one of the following parameters: amount of received basic 
media data units; amount of basic media data units belonging to predefined basic 
media data unit sequences; reception rate of received basic media data units; 
reception rate of basic media data units belonging to predefined basic media 
data sequences; and the identity of received basic media data unit sequences. 
[0066] Although the present invention is described predominantly in terms of 
the transmission and storage of video and audio information encoded in 
accordance with the MPEG format, the concepts and methods are broad enough 
to encompass storage and distribution systems using other data compression 
techniques and other data formats. Throughout this detailed description, 
numerous details are specified such as data stream structures, in order to 
provide a thorough understanding of the present invention. For example, it is 
assumed that a sequence of basic media data units is an MPEG compliant 
program including at least one MPEG compliant elementary stream, and that the 
basic media data units are macroblocks. Each video elementary stream includes 
P-pictures, B-pictures and l-pictures grouped in group of pictures (GOP). 
[0067] Referring to Figure 1 illustrating a statistical multiplexer 12 and its 
surroundings, in accordance to a preferred embodiment of the invention. 
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Statistical multiplexer 12 is coupled to digital storage medium 8, satellite receiver 
1 1 and a compressed media source 10, such as an MPEG compliant encoder, 
for receiving programs. Statistical multiplexer 12 is also coupled to control 
interface unit 1 1 for receiving information such as external modification priority 
information. Statistical multiplexer 12 has an output port for providing a 
multiplexed sequence, such as a transport stream. The output port is coupled to 
a communication channel 7 that has an available bandwidth for conveying the 
transport stream. 

[0068] Controllable analyzer 13 is coupled to communication module 7 and to 
statistical multiplexer 12, for analyzing basic media data units, modified basic 
media data units, groups of basic media data units, groups of modified basic 
media data units, a portion of the transport stream, and the like. Optionally, 
controllable analyzer 13 is also configured to measure the available bandwidth of 
communication channel 7. Statistical multiplexer 12 is configured to send 
controllable analyzer 13 control signals for determining which basic media data 
units, modified basic media data units, groups of basic media data units, groups 
of basic media data units, or portion of the transport stream to analyze. 
[0069] Conveniently, controllable analyzer 13 has a quality measurement unit, 
for measuring a quality of at least one basic media data unitat a time. 
Conveniently, controllable analyzer can measure a media picture, including a 
plurality of basic media pictures at a time. Preferably, the media pictures are 
MPEG compliant and the quality measurement is based upon an analysis of 
MPEG artefacts. Controllable analyzer 13 can include at least one quality 
measurement unit such as Rhode & Schwartz DVQ or Tektronix quality of 
service monitor PQM300. These quality analyzers are configured to analyze 
some programs out of a larger number of programs within the transport stream. 
The selection of the analyzed programs can reflect various parameters, such as 
a predefined program priority, amount and extent of program modifications during 
a predefined time period, and the like. The selection can also be random, 
arbitrary or of a predefined pattern. Conveniently, the results of the analysis 
provide relatively long-term statistics and can be used to adjust the modification 
priorities of various programs. 
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[0070] Conveniently, the basic media data units are MPEG compliant and the 
modification can be implemented by at least one of the following methods: 
removing filler pictures; removing stuffing bits; selectively setting DCT 
coefficients to zero; discarding data used to represent selected media pictures; 
discarding data used to represent selected media pictures and generating repeat 
information in the bit stream such that a decoder can repeat the dropped 
pictures; re-quantizing quantized DCT coefficients; extracting and changing the 
quantization scale factors; decoding and encoding at different bit rates; and 
changing the resolution of a video image. 

[0071] Referring to Figure 2, illustrating statistical multiplexers such as 
statistical multiplexer (SM-H) 27 and statistical multiplexers (SM-s) 36-1 - 36-S, 
that are coupled to a plurality of sets of service groups, constructed and 
operative in accordance with preferred embodiments of the present invention. A 
local distribution center 20, also referred to as headend 20, is configured to 
receive information signals from distribution satellites 14 and 15, via satellite 
dishes 24, 22 and to receive signals from headend content provider 28. These 
information signals include a plurality of MPEG compliant programs. Local 
distribution center 20 selects some of the received signals and combines the 
selected signals to form a multiplexed signal such as MUX_SIGNAL 38-1 - 38-S. 
[0072] Although not illustrated in Figure 2, local distribution center 20 and 
hubs are further configured to receive upstream signals. These upstream signals 
can include external modification priority information that reflects end-users 
preferences and the like. 

[0073] Each service group set 37-s includes r service groups 35-s-r, each 
service group 35-s-r includes q end-users 34-s-r-q. Index r ranges between 1 and 
R, index s ranges between 1 and S, and index q ranges between 1 and Q. 
[0074] SM-H 27 is functionally located within local distribution center 20 and 
each of SM-s 36-s is functionally located within each hub. SM-s 36-s is coupled 
to the s'th set of service groups 37-s, via broadband networks, such as HFC, 
DSL networks and the like, and is configured to provide each service group 35-s- 
r of the s'th set of service groups 37-s a distinct content. For convenience of 
explanation the multiplexed signal provided to the r'th service group of the s'th 
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set of service groups is denoted MUX_SIGNAL 40-s-r. MUX_SIGNAL 40-s-r 
includes a plurality of packets such as MPEG compliant transport packets to be 
provided to the service group 35-s-r during at least one session. 
[0075] SM-H 27 is configured to provide each of SM-s 36-s a distinct 
multiplexed signal, denoted MUX_SIGNAL-s. MUX_SIGNAL-s includes a 
plurality of packets such as MPEG compliant transport packets to be provided to 
set of service groups 37-s during at least one session. Each of SM-s 36-s is 
configured to receive MUX-SIGNAL-s 38-s, and to select out of MUX_SIGNAL-s 
38-s a plurality of multiplexed portions, each portion to be provided to a distinct 
service group out of the R service groups of the s'th set of service groups 
coupled to SM-s 36-s . SM-s 36-s are usually further adapted to receive 
additional signals, such as data signals, media signals, advertisement signals 
and the like from local content providers, such as hub content provider (not 
shown) and to add the additional signals to the multiplexed portions to provide 
MUX-SIGNAL 40-s-r. MUX-SIGNAL 40-s-r includes a plurality of packets such as 
MPEG compliant transport packets. 

[0076] Each of statistical multiplexers 36-s and 27 can further be coupled to 
controllable analyzer, digital storage medium and a control interface unit such as 
controllable analyzer 13, digital storage medium 8 and a control interface unit 11 
of Figure 1 . 

[0077] A statistical multiplexer can have various configurations, such as 
centralized or a parallel configurations and the like. In a centralized configuration, 
various processing steps, such as the calculation of a modification priority, the 
modification of basic media data units and the provision of a multiplexed 
sequence to a communication module are executed by a control unit, and a 
modification unit. The statistical multiplexer includes (a) a control unit, (b) at least 
one input, coupled to the control unit, for receiving at least one basic input data 
unit sequence, (c) an output, coupled to the control unit and to a communication 
module, for providing a multiplexed sequence to the communication module, (d) 
a modification unit, coupled to control unit, to the at least one input and to the 
output, the modification unit is configured to modify selected basic media data 
units to provide corresponding basic media data units, in response to control 
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units from the control unit; wherein a modified selected basic media data unit is 
smaller than the corresponding selected basic media data unit. The control unit is 
configured to: (i) determine a modification priority of each basic media data unit 
of the received at least one basic media data unit sequence; (ii) select basic 
media data units to be modified, in response to the modification priority and to an 
available bandwidth of the communication module; (iii) control the provision of 
the selected basic media data units to the modification unit and the modification 
of each of the selected basic media data units; and (iv) control the provision of a 
multiplexed sequence including the modified selected basic media data units and 
non-selected basic media data units to the communication channel. 
[0078] In a distributed configuration, each sequence of basic media data unit 
is processed by a single sequence processor, and the various basic media data 
units are distributed among the plurality of sequence processor by a centralized 
distribution unit. A distributed configuration is illustrated at Figure 6. It is 
assumed, for convenience of explanation only that the basic media data units are 
MPEG compliant, that each sequence of basic media data units has a distinct 
PID, and that a basic media data unit is a macroblock. 

[0079] Portion 200 includes a PID unit 202, channel simulator 204, packetizer 
and output multiplexer 206. A plurality (F) of sequence processors 220-f are 
coupled between PID unit 202 and channel simulator 204 and between PID unit 
202 to packetizer and output multiplexer 206. Each sequence processor 220-f 
includes data analyzer 210-f, modification unit 216-f, processor and decoder 
simulator 214-f and a selector 218-f. Index f ranges between 1 and F, F is a 
positive integer that either exceeds the amount of possible programs or equals 
said amount. The data paths and control and status paths are illustrated by two 
types of arrows, the latter are illustrated by dashed lines, while the former are 
illustrated by solid lines. 

[0080] PID unit 202 is conveniently preceded by an assembly unit (not shown) 
that receives packet such as transport packets and provides PID unit 202 media 
segments of elementary streams that are included within the packets payloads. 
PID unit 202 receives the segments of the elementary streams and provides 
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each elementary stream to a sequence processor in view of the PID of the 
elementary stream segment. 

[0081] First sequence processor 220-1 includes data analyzer 210-1 , 
modification unit 216-1, processor and decoder simulator 214-1 and selector 
218-1 . First sequence processor 220-1 receives segments of a predefined 
elementary stream. It is assumed that the elementary stream segments include a 
plurality of macroblocks and associated data that are arranged as l-pictures, B- 
pictures and P-pictures. Each macroblock is provided to: (a) data analyzer 210-1, 
to ;be analyzed, (b) modification unit 216_1 , to generate at least one 
corresponding modified basic media data unit; and to (c) selector 218_1, that 
eventually provides either the macroblock or a corresponding modified 
macroblock to packetizer and output multiplexer 206. Data analyzer 210-1 also 
receives and accordingly analyzes modified macroblocks. 
[0082] Data analyzer 201-1 is coupled to (i) PID unit 202, (ii) modification unit 
216_1 and (iii) processor and decoder simulator 214_1 for (i) receiving segments 
of the elementary stream, (ii) receiving modified macroblocks, and (iii) providing 
to processor and decoder simulator 214_1 control and status information, such 
as macroblock and modified macroblock size, macroblock and modified 
macroblock quantizer scale, timing information related to the macroblock, a 
dependency of other macroblocks on a received macroblock, picture type, end 
and beginning of slices and pictures and the like. 

[0083] Processor and decoder simulator 214_1 is coupled to modification unit 
216_1 for determining which macroblocks to modify, and optionally in which 
manner. Processor and decoder simulator 214_1 is also coupled to channel 
simulator 204 for (i) providing status and control information such as size of 
macroblocks or modified macroblocks to be provided to communication channel 
7, timing information of the macroblocks or modified macroblocks, modification 
priority of macroblocks, and for (ii) receiving information indicating an amount or 
the identity of macroblocks or modified macroblocks that can be transmitted over 
communication channel. Channel simulator 204 receives a bandwidth indication 
representing an available bandwidth of communication module 7 and determines 
which macroblocks/ modified macroblocks can be provided to the communication 
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module. Processor and decoder simulator 214_1 is configured to receive external 
modification priority information, if said information is relevant to the first 
elementary stream. Processor and decoder simulator 214J is also coupled to 
selector 218_1 for selecting whether to provide a macroblock or a modified 
macroblock to packetizer and output multiplexer 206. Packetizer and output 
multiplexer 206 packetizes the received macroblocks to generate transport 
packets and multiplexes the macroblocks and modified macroblocks sent from 
selectors 218_1 - 218_F to communication channel 7. 
[0084] Referring to Figure 3 illustrating a method 100 for generating a 
multiplexed sequence of basic media data units to be transmitted over a 
communication medium. 

[0085] Method 100 starts by step 102 of defining a modification priority 
function. Step 102 allows for a selection between various combinations of at least 
one of the following parameters of a basic media data unit: (a) a quality of the 
basic media data unit; (b) quality degradation of the basic media data unit; (c) 
compression level of the basic media data unit; (d) a dependency of at least one 
other basic media data unit upon the basic media data unit; (e) a combination of 
qualities of basic media data units that belong to a same basic media data unit 
group as the basic media data unit; (f) a combination of quality degradations of 
basic media data units that belong to a same basic media data unit group as the 
basic media data unit; (g) a combination of compression levels of basic media 
data units that belong to a same basic media data unit group as the basic media 
data unit; (h) size of the basic media data unit; (i) dependency of at least one 
other basic media data unit on the basic media data unit; (j) combination of 
dependencies of other basic media data units on basic media data units that 
belong to the same basic media data unit group as the basic media data unit. 
Conveniently, step 102 also includes a determination of the allowable range of 
each parameter. Assuming that the basic media data unit is a macroblock, and 
that macroblocks are grouped in pictures, the quantizing level of the macroblock 
can reflect the quality degradation of a macroblock and the compression level of 
the macroblock. The dependency between pictures can be learnt from their type 
(l-picture, P-picture and B-picture), an amount of temporal difference information 
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contained within each picture, and the content of the picture itself, such as the 
content of the motion vectors. The quantization level can also reflect the quality 
of each macroblock. The quality can also be selectively measured by 
measurement units such as controllable analyzer 13 of Figure 1 . 
[0086] For example, a modification priority MP x ,z of the x'th basic media data 
unit of the z'th basic media data unit sequence (BM X( z) can be equal to (5+EMP z 
- PTx,z)*QSx,z , EMP Z reflects an external modification priority given to the z'th 
sequence of basic media data units, PT X( z reflects a dependency of at least one 
other basic media data unit upon BM Xi2 and QS x ,z is the quantization level of 
BM X)Z . In such a case the variance of quantizer levels of received basic media 
data units is larger than the variance of the quantizing levels within the 
multiplexed sequence. 

[0087] Conveniently, the modification priority function can be changed 
dynamically, and step 102 can include defining the possible allowable changes. 
Accordingly step 102 can include defining a plurality of modification priority 
functions and when to apply each of said functions. 

[0088] Step 102 is followed by step 104 of receiving external modification 
priority information. The external modification priority information can be provided 
by various entities located along a logical path extending from a generation entity 
of the basic media data units, at least one multiplexing or even re-multiplexing 
entity to end-users. Referring to the example set forth in Figure 2, SM-1 36-1 can 
receive external modification priority information from a hub controller (not 
shown), from at least one end-user 34-1 -r-q, from at least one service group 35- 
1-r and can also process the behavior pattern or preferences of more than a 
single end-user or more than a single service group 35-1 -r to generate external 
modification priority information. SM-H 27 can receive external priority 
information from local content provider 28, from the operator of local distribution 
center 20 and can further receive such information from end-users, service 
groups, sets of service groups and hubs. SM-H 27 can also process the behavior 
pattern of end-users, service groups and sets of service groups to produce the 
external priority information. Each statistical multiplexer can receive external 
modification priority information from a higher entity, such as from a media 

25 



stream generator or provider responsive to the provision of signals via 
distribution satellites 14 and 15. Each statistical multiplexer out of SM-s 36-s and 
SM-H 27 can be further adapted to filter received transport packet in view of the 
transport packet program. The programs entity can be learnt from the PID 
embedded within each transport stream packet. 

[0089] Step 104 is followed by step 106 of receiving a plurality of basic media 
data units belonging to at least one basic media data unit sequence. Referring to 
the example set forth in the upper part of Figure 5, a plurality of basic media data 
units referenced 50 it j, are received during P consecutive time slots. Index i 
denotes a basic media data unit and index j denotes a basic media data unit 
sequence index. Index i ranges between 1 and either M,N,0 or P, index j ranges 
between 1 and K. It is assumed, for convenience of explanation only, that K basic 
media data unit sequences are provided in parallel to statistical multiplexer 12 
over a plurality of communication channels and that packets that include up to K 
basic media data units can be provided during a single time slot of a predefined 
length. The first basic media data unit sequence includes M basic media data 
units referenced 50i,i - 50i iM that were received during M consecutive time slots. 
The second basic media data unit sequence includes P basic media data units 
referenced 502,1 - 502, p that were received during P consecutive time slots. The 
third basic media data unit sequence includes L basic media data units 
referenced 503,1 - 503, L that were received during L consecutive time slots. The 
K'th basic media data unit sequence includes N basic media data units 
referenced 50i,i - 50i (N that were received during N consecutive time slots. 
L<M<N<P. Conveniently, step 106 includes analyzing at least some of the 
characteristics of the basic media data units to provide an analysis that can 
change the modification priority function. 

[0090] Step 106 is followed by step 108 of estimating the current size or bit 
rate of the multiplexed sequence. The size of the multiplexed sequence must 
match an available memory space of a digital storage medium or, as illustrated in 
Figures 1,2 and 5, an available bandwidth of a communication medium. Methods 
and systems for estimating an available bandwidth are known in the art. Briefly, 
the available bandwidth can either be measured or known in advance. In cases 
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where the available bandwidth is fixed, step 110 can be skipped, as long as the 
fixed available bandwidth is known to method 100. Step 108 can also include a 
step of changing the modification priority function in view of the characteristics of 
the received basic media data units. 

[0091] Step 1 08 is followed by step 1 1 0 of determining a modification priority 
of the received basic media data units. Step 108 includes applying the 
modification priority scheme that was defined in step 102 on the received basic 
media data units. 

[0092] Step 1 10 is followed by step 112 of selecting basic media data units to 
be modified, in view of their modification priority and optionally in view of the 
current size/ bit rate of the multiplexed sequence. 

[0093] Step 1 12 is followed by step 1 14 of modifying selected basic media 
data units to provide corresponding modified basic media data units. This step 
includes modifying basic media data units, starting with the basic media data 
units of the highest modification priority and continuing to lower modification 
priority basic media data units while constantly tracking the aggregate size of 
modified and not modified basic media data units to assure that the aggregate 
size does not exceed the size estimated in step 108. Conveniently, the constant 
tracking allows stopping the modification after the aggregate size either matches 
the required aggregate size or is just slightly below the required aggregate size. 
[0094] Step 1 14 is followed by step 1 16 of multiplexing the modified selected 
basic media data units to provide the multiplexed sequence. Referring to the 
example set forth in the lower part of Figure 5, in which the content of a 
multiplexed sequence is illustrated, a plurality of modified and not modified basic 
media data units form a multiplexed sequence. The aggregate size of modified 
and not modified basic data units forming the multiplexed sequence at time slots 
T1 - T p are denoted I size(Ti) - I size(T p ) respectively. During time slot T1 a 
plurality of basic media data units are provided to a communication module. 
Accordingly, the multiplexed sequence includes basic media data units 50 2l i, and 
modified basic media data units 50\i and 50\i. The implementation of lossless 
techniques is illustrated by basic media data unit 50 3t i that is delayed and is 
transmitted along with basic media data units and modified basic media data 

27 



units that were received after it. Basic media data units and modified basic media 
data units of the first sequence are transmitted during Ti - T M . Basic media data 
units and modified basic media data units of the second sequence are 
transmitted during Ti - T p . Basic media data units and modified basic media data 
units of the third sequence are transmitted during T2 - Tl+i. Basic media data 
units and modified basic media data units of the fourth sequence are transmitted 
during T1 - Tn. 

[0095] Step 1 16 is followed by step 120 of transmitting the multiplexed 
sequence over a communication media. It is noted that although step 120 
illustrates a transmission of the multiplexed sequence, and preferably a 
transmission to a remote location, step 116 can be followed by a step of storing 
the multiplexed sequence in a digital storage medium. Conveniently, the size of 
the multiplexed sequence has to match the available space of the digital storage 
means. 

[0096] Step 1 1 6 is also followed by step 1 1 8 that is followed by step 1 06. Step 
118 includes analyzing a portion of the multiplexed sequence to provide analysis 
information. Referring to the example set forth in Figure 1, step 118 can include 
an analysis of a portion of the multiplexed sequence by a quality measurement 
unit of controllable analyzer 13. The selection of the analyzed programs can 
reflect various parameters, such as a predefined program priority, amount and 
extent of program modifications during a predefined time period, and the like. 
The results of the analysis provide relatively long-term statistics and can be used 
to adjust the modification priorities of various programs. 
[0097] Referring to Figure 4 illustrating method 200 for generating and 
transmitting a multiplexed sequence over a communication channel, according to 
a preferred embodiment of the invention. The multiplexed sequence includes at 
least one basic media data unit sequence. Method 200 is analogues to method 
100 but steps 114 and 116 are replaced by steps 122, 124 and 126. 
[0098] According to an aspect of the invention the comparison involves 
applying a comparison function. The comparison function can be changed 
dynamically, and steps 102 and 106 of method 200 can be altered. Step 102 can 
include defining the possible allowable changes of the comparison function. Step 
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102 can include defining a plurality of comparison functions and when to apply 
each of said functions. Step 106 includes analyzing at least some of the 
characteristics of the basic media data units to provide an analysis that can 
change the comparison priority function. 

[0099] Step 122 includes modifying each of the selected basic media data 
units to provide corresponding modified basic media data units. Conveniently, 
step 122 does not include a step of constantly tracking the aggregate size of 
modified and not modified basic media data units to assure that the aggregate 
size does not exceed the size estimated in step 108. 
[00100] Step 122 is followed by step 124 of replacing selected basic media 
data units with the corresponding modified basic media data units in response to 
a comparison between the basic media data units and the corresponding 
modified basic media data units to provide replacing basic media data units. 
According to an aspect of the invention the comparison between a basic media 
data unit and a corresponding modified basic media data unit can include 
comparing between a combination of at least one of the following parameters: (a) 
quality; (b) combination of qualities of basic media data unit group members; (c) 
compression level; (d) combination of compression levels of basic media data 
unit group members; (e) temporal difference information; (f) combination of 
temporal difference information of basic media data unit group members; (g) size 
of the basic media data unit and the corresponding basic media data unit. 
According to another aspect of the invention the replacement of selected basic 
media data units by corresponding basic media data units is responsive to a 
comparison between a basic media data unit, a corresponding modified basic 
media data unit; another basic media data unit, another corresponding basic 
media data unit. The comparison can include comparing between a combination 
of at least two of the following parameters: (a) quality; (b) combination of qualities 
of basic media data unit group members and combination of qualities of other 
basic media data unit group members; (c) compression level; (d) combination of 
compression levels of basic media data unit group members and combination of 
compression levels of other basic media data unit group; (e) temporal difference 
information; (f) combination of temporal difference information of basic media 
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data unit group members and combination of temporal difference information of 
other basic media data unit group members; and (g) size. 
[00101] According to an aspect of the invention, step 124 includes a step of 
constantly tracking the aggregate size of the replacing and non-replacing basic 
media data units to assure that the aggregate size of the multiplexed sequence 
does not exceed the size estimated in step 108. Conveniently, the constant 
tracking allows stopping the replacement after the aggregate size either matches 
the required aggregate size or is just slightly below the required aggregate size. 
[00102] Step 124 is followed by step 126 of multiplexing the replacing basic 
media data units and basic media data units that were not replaced to provide 
the multiplexed sequence. 

[00103] It will be apparent to those skilled in the art that the disclosed subject 
matter may be modified in numerous ways and may assume many embodiments 
other then the preferred form specifically set out and described above. 
[00104] Accordingly, the above disclosed subject matter is to be considered 
illustrative and not restrictive, and to the maximum extent allowed by law, it is 
intended by the appended claims to cover all such modifications and other 
embodiments, which fall within the true spirit and scope of the present invention. 
[00105] The scope of the invention is to be determined by the broadest 
permissible interpretation of the following claims and their equivalents rather then 
the foregoing detailed description. 
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